In recent years, three-dimensional (3D) imaging has found various applications, such as virtual-reality visualization, manufacturing, machine vision, surgical models, authentication, etc. The 3D images may be captured using a regular camera for the texture information and a separate depth camera (e.g. Time of Flight camera) for the depth information of objects in the scene in the field of view. The 3D images may also be captured using multiple cameras, where multiple cameras are often used in a planar configuration to capture a scene from different viewing angles. Point correspondence is then established among multiple views for 3D triangulation.
Another 3D imaging technology, named structured light technology, has been developed to derive the depth or shape of objects in the scene using a single camera. In the structured light (SL) system, one or more light sources and a projector are often used to project known geometric pattern(s) onto objects in the scene. A regular camera can be used to capture images with and without the projected patterns. The images captured with and without the structured light can be used to derive the shapes associated with the objects in the scene. The depth or shape information is then used for the regular images, which are captured without structured light, to create 3D textured model of the objects. The structured light technology has been well known in the field. For example, in “Structured-light 3D surface imaging: a tutorial” (Geng, in Advances in Optics and Photonics, Vol. 3, Issue 2, pp. 128-160, Mar. 31, 2011), structured light technology using various structured light patterns are described and the corresponding performances are compared. In another example, various design, calibration and implement issues are described in “3-D Computer Vision Using Structured Light: Design, Calibration and Implementation Issues” (DePiero et al., Advances in Computers, Volume 43, Jan. 1, 1996, pages 243-278). In U.S. Pat. No. 8,493,496, issued on Jul. 23, 2013, a method and apparatus for mapping an object are disclosed. According to U.S. Pat. No. 8,493,496, a transparency containing a plurality of micro-lenses is arranged in a non-uniform pattern. A light source, which is configured to trans-illuminate the transparency with optical radiation and the micro-lenses are configured to focus the optical radiation to form, at a focal plane, respective focal spots in a non-uniform pattern. An image sensor captures an image of the pattern that is projected onto the object for reconstructing a 3D map of the object. The details of the structured light technology are well-known in the field and therefore the details are not repeated here.
Recently, structured light imaging has been used for facial recognition as an authentication method for a user to unlock a mobile device such as a smart phone. The structured light 3D system is often intended for mapping an object in a static environment, where the object is stationary. Furthermore, in order to derive reliable 3D model, the structured-light images are often captured using a structured light at much higher intensities than the ambient light. Therefore, the conventional structured light imaging approach may not suitable for the 3D facial recognition in mobile device since the strong structured light is not only disturbing, but also raises eye safety concerns. In order to overcome the issues, a system introduced to the market uses dedicated camera to capture structured-light images. Furthermore, near-infrared light sources are used to project the structured-light patterns to avoid or reduce disturbance to the subject during structured-light image capture. For example, iPhone X™ recently introduced by Apple Inc.™ incorporates a structured-light transmitter using a VCSEL (vertical-cavity surface-emitting laser) as a light source to project 30,000 dots onto an object (Zac Hall, “iPhone X's one design limitation rumored to be improved next year”, 9to5Mac Online Article, Jan. 16, 2018, https://9to5mac.com/2018/01/16/iphone-12-almost-notchless/). A structured light receiver comprising a 1.4 MP CMOS (complementary metal-oxide-semiconductor) sensor with a near-infrared filter is used to capture structured light images. Furthermore, iPhone X™ includes a flood illuminator (Alex Webb and Sam Kin, “Inside Apple's Struggle to Get the iPhone X to Market on Time”, Bloomberg Technology, Oct. 25, 2017, https://www.bloomberg.com/news/articles/2017-10-25/inside-apple-s-struggle-to-get-the-iphone-x-to-market-on-time+&cd=1&hl=en&ct=clnk&gl=us), which beams an infrared light for the infrared camera to establish the presence of a face. While the use of a separate sensor and light source in the non-visible light spectrum provides a reliable means for capturing structured light images, the solution is quite costly due to the additional components required (i.e., the dot projector/VCSEL light source, the flood illuminator and the infrared camera). FIG. 1 illustrates an example of the mobile phone 100 in the market with the structured light for face recognition, where the dot projector/VCSEL light source 110, an infrared camera 120, a flood illuminator 130 and a front camera 140 are shown.
For any consumer application, the cost is a very sensitive factor to take into consideration. It is desirable to develop 3D structured light imaging systems with reduced components while maintaining the quality and accuracy as the system with a separate structured light projector and separate structured light image sensor, and without causing noticeable disturbance to the subject.